evalu challeng direct inform dseek support system deciph mobil search pattern studi yahoo mobil search queri improv activ learn recal disjunct boolean constraint perform threshold practic text classif practic classif mix learnabl unlearn class classifi minimum perform threshold deploi problem exacerb train set creat activ learn bia activ learn train set make hard determin class learn give evid gener effici method reduc bia correctli identifi class learn character number scenario activ learn succe difficulti link spam detect base mass estim link spam intend mislead search engin trigger artifici high link base rank specif target web page paper introduc concept spam mass measur impact link spam page rank discuss estim spam mass estim identifi page benefit significantli link spam experi host level yahoo web graph spam mass estim successfulli identifi ten thousand instanc heavi weight link spam autom gather web inform ddepth examin agent interact search engin paper refer spider softbot meta search applic autom inform gather process agent multitask web search session effici pagerank approxim graph aggreg present framework approxim random walk base probabl distribut web page graph aggreg partit web graph class quasi equival vertic project page base random walk approxim class comput stationari probabl distribut result class base random walk distribut quickli reconstruct distribut page framework approxim pagerank distribut set class set page web host experi web graph billion page produc rank spearman rank order correl respect pagerank simplist implement method requir half run time highli optim implement pagerank impli larger speedup factor internet search engin past futur search engin shape web tempor comparison altavista web search major web search engin altavista essenti tool quest locat onlin inform articl report research transact log analysi examin characterist altavista web search occur research question examin altavista web search current characterist altavista search includ durat frequenc search session inform altavista user occur result research show move interact increas session queri length session durat minut frequenc interact increas happen quickli broaden rang web searcher inform frequent term account total term usag discuss implic find develop web search engin multitask web search alta vista combat web spam trustrank web spam page techniqu achiev higher deserv rank search engin result human expert identifi spam expens manual evalu larg number page propos techniqu semi automat separ reput good page spam select small set seed page evalu expert manual identifi reput seed page link structur web discov page good paper discuss wai implement seed select discoveri good page present result experi run world wide web index altavista evalu perform techniqu result show effect filter spam signific fraction web base good seed set site effici pagerank approxim graph aggreg effect special multimedia collect web search monster gate softbot visit web search engin analysi multimedia search altavista analysi multimedia search altavista phrase recognit expans short paper examin question queri pars world wide web queri present method phrase recognit expans train corpu approxim million web queri handwritten context free grammar em algorithm estim paramet probabilist context free grammar pcfg system develop carrol pcfg comput probabl pars user queri reflect linguist structur word usag domain pars optim syntact pars user queri obtain emploi phrase recognit expans phrase recognit increas retriev precis phrase expans appli make short web queri exploit hierarchi text categor metadata mix dmedia access compar studi featur select text categor paper compar studi featur select method statist learn text categor focu aggress dimension reduct method evalu includ term select base document frequenc df inform gain ig mutual inform mi test chi term strength ts found ig chi effect experi ig threshold knearest neighbor classifi reuter corpu remov remov uniqu term yield improv classif accuraci measur averag precis df threshold perform similarli found strong correl df ig chi valu term suggest df threshold simplest method lowest cost comput reliabl ig chi comput measur expens ts compar favor method vocabulari redu space optim total rank real life inform retriev commerci search engin panel dconstant dtime cluster arbitrari corpu subset method exist constant time cluster corpu subset select scatter gather brows paper expand techniqu give algorithm almostconst time cluster arbitrari corpu subset algorithm slower cluster document set scratch medium size larg set significantli faster algorithm cluster arbitrari subset larg corpora obtain instanc boolean search quickli interact set introduct document cluster emerg import tool present navig document collect scatter gather brows paradigm cluster document topic coher group present descript textual summari user inform summari user select cluster form subcollect iter examin cluster reclust veriti trec dof dthe dbox cooccurr dbase thesauru applic inform retriev method combin document filter strong empir theoret evid combin retriev method improv perform paper systemat compar combin strategi context document filter queri tipster refer corpu find simpl averag strategi improv perform direct averag probabl estim correct approach probabl estim renorm logist regress relev judgement examin complex combin strategi find success due high correl filter method optim train data emploi similar document represent introduct text filter system monitor incom document stream select document identifi relev queri profil profil interact reduc number independ bina reexamin cluster hypothesi scatter fgather retriev result present scatter gather cluster base document brows method altern rank titl organ view retriev result systemat evalu scatter gather context find signific improv similar search rank result evid valid cluster hypothesi state relev document tend similar relev document describ system emploi scatter gather demonstr user system close full potenti introduct import servic offer inform access system organ retriev result convent system rank result base automat assess relev queri altern includ graphic displai interdocu similar relationship fix attribut queri term distribut pattern xerox trec site report rout filter nlp spanish track report divid section section describ work rout filter hull schutz pedersen section cover nlp track grefenstett schulz gaussier final section address spanish track grefenstett schulz hull rout filter reveal collect structur inform access interfac comparison classifi document represent rout problem paper compar learn techniqu base statist classif tradit method relev feedback document rout problem classif techniqu decis rule deriv explicit error minim linear discrimin analysi logist regress neural network demonstr classifi perform relev feedback rocchio expans trec trec rout task error minim difficult high dimension featur space converg process slow model prone overfit strategi latent semant index optim term select reduc number featur result featur base latent semant index effect techniqu linear discrimin analysi logist regress protect overfit neural network perform equal set featur advantag addit inform featur set input trainabl document summar summar reduc complex length retain essenti qualiti origin paper focuss document extract kind comput document summari xerox site report trec track document sampl expect chanc term select binomi likelihood ratio test compar occurr document occurr rest collect select term weight proport signific occurr sampl document baselin result run affect program error queri set base infl infl np expand tabl averag precis relev doc averag precis doc spanish queri correct spanish perform figur present tabl includ run base stop list morpholog analysi infl text lemmat stem inflect morpholog infl np noun phrase weight doubl expand queri expans uncorrect perform figur infl np expand submit run present separ result queri trec trec substanti longer note result reflect differ length find lemmat inflect morpholog help case make absolut differ perform queri long user examin fewer document improv conclus agre result obtain english spanish inflect morpholog effect english counterpart doubl rich interact digit librari articl cross cut number usual layer digit librari architectur includ retriev engin access protocol intermedi servic user inter rose kearn xerox allair hick love magnin reid kobayashi adam chamber howard vairo montgomeri merritt steven hajia rosner galvin biel lovel bardslei rosner senter sachdav rosen carei male mcoi filt molin cut bun car puc yea kei ran mag pit nu sai rei bar ro ken mer ish kob spencer kearn ceo chairman chief execut offic figur hyperbol tree browser face retriev engin support broader rang oper user understand properti entir collect group document document effect support queri refin stage task access protocol intermedi server provid form bridg meta servic explicitli reveal characterist inform mediat access mechan allow effect memor render sourc content final user interfac thought complet workspac varieti tool flexibl interact richli convei inform cooccur dbase thesauru applic inform retriev xerox trec report combin exact fuzzi predictor report section describ rout experi section describ ad hoc experi section discuss result futur experi metadata mix dmedia access constant interact dtime scatter fgather brows larg document collect scatter gather document brows method fast document cluster produc tabl contentslik outlin larg document collect previou work develop linear time document cluster algorithm establish feasibl method moder larg collect linear time algorithm slow support interact brows larg collect tipster darpa standard text retriev evalu collect present scheme support constant interact time scatter gather arbitrarili larg collect nearlinear time preprocess involv construct cluster hierarchi modif scatter gather emploi scheme tipster collect present background previou work scatter gather shown document cluster class tool brows larg text collect brows distinguish sea practic part dof dspeech tagger present implement part speech tagger base hidden markov model methodolog enabl robust accur tag resourc requir lexicon unlabel train text requir accuraci exce describ implement strategi optim result high speed oper applic tag phrase recognit word sens disambigu grammat function assign scatter fgather cluster dbase approach brows larg document collect document cluster receiv inform retriev tool object fall main categori cluster slow larg corpora run time quadrat number document cluster appreci improv retriev argu problem aris cluster attempt improv convent search techniqu cluster inform access tool obviat object power access paradigm present document brows techniqu emploi document cluster primari oper present fast linear time cluster algorithm support interact brows paradigm introduct document cluster extens investig methodolog improv document search retriev excel review gener assumpt mutua object dorient architectur text retriev aspect inform access system case optim composit function hotli debat applic scenario put demand individu compon essenc quickli build system permit explor design implement strategi paper present softwar implement architectur text retriev system facilit function modular mix match combin modul implement definit inter modul protocol show object orient approach easili accommod type architectur design principl exemplifi code exampl common lisp code exampl constitut oper retriev system design principl protocol implement instanti larg scale retriev prototyp research laboratori introductio optim dynam invert index mainten free text search rapidli evolv corpora dynam updat invert indic basic requir tree effect tool implement indic zipfian distribut post suggest space time optim uniqu task present optim merg updat perform straight forward block updat puls significantli reduc space requir sacrif perform invert indic standard free text search method inform retriev ir implement effici invert index includ standard boolean extend boolean proxim relev search algorithm invert index data structur map word atom search item set document set index unit word post individu post binari indic presenc word document 